1. Technical Field
The present invention is generally related to flight instrumentation, and more particularly to the display of flight information in a cockpit of an aircraft.
2. Background of the Invention
The visual cues used for situational awareness in ordinary activities and in clear-day flight are continuously obtained without training or conscious attention, and processed by a complex native neural processing network that can detect motion instantly without fixing on the moving object, and can then, in parallel, break complex images down into elements that are then processed separately. The first component of this system is the neural network covering the surface of the retina itself, in which signals from adjacent rods and cones are instantly inter-compared to detect motion anywhere in the entire field of vision without eye movement. Movement-detection signals and visual image data are then transmitted to the brain separately. The image data is further broken down and evaluated in separate brain regions. For example, detection and evaluation of parallelism between lines, or evaluation of geometric shapes including recognition of faces or instrument dials, are all done separately. The movement data and processed image data are then continuously coordinated to produce a largely subconscious dynamic mental image of position and movement in space. This occurs rapidly, requires no adult learning, and is directly linked to rapid responses, for example blinking the eye, or shielding the face to protect against a visually detected assault, or continuous maintenance of an upright position while walking through diverse environments. These responses persist in the face of stress, nausea, and disorientation. Much of this circuitry is hard wired, rapid, automatic, subconscious, and unlearned, and appears to be under genetic control.
In contrast, current flight instruments present processed data to the pilot. For example, movement with respect to world coordinates (climbing, diving, turning) are not indicated by moving points or objects, but are presented as rates of climbing, diving or turning, often indicated by a number, or needle or cursor positions. These have no analogues in real life. Our bodily sensors are designed, in contrast, to detect movement and not rates of movement. A basic assumption has been that images of individual instruments ought to be processed in the same ways, and at the same rates as the environmental inputs and conditions they represent. Long training and concentrated attention are required for instrument flight, which is extremely fatiguing under adverse conditions. Instrument flight is a learned skill, which can rapidly degenerate under conditions of extreme stress, terror, nausea, and disorientation. Thus instrument flight image processing is through a complex learned program that is easily degraded, and under the best conditions, requires more processing time than does the analogous natural process in a natural environment. It is evident that if the visual elements of a normal environment could be abstracted and their basic elements continuously presented in the cockpit environment, instrument flight could be vastly improved, done with less fatigue, with more attention to other flight tasks, and with much less training.
The objective of this invention, therefore, is to provide the same cues used in nature to provide continuously those visual elements required for situational awareness in flight or in other forms of transportation involving disorienting motion. An essential concept of the invention is to provide moving, largely peripheral, objects, icons, or points in space that mimic the essentials of natural experience, which essentials are acquired without conscious attention, with minimal learning, and would be always present in a flight environment. An additional objective is to provide a simple and ultimately inexpensive flight orientation system for small aircraft and gliders flown by non-professional pilots.
Orientation relative to the outside world, balance, rate and direction of movement are synthesized from efferent visual and vestibular signals, which in some cases can be in conflict. For example, when a pilot in a so called xe2x80x9cgraveyard spiralxe2x80x9d breaks out of a cloud into clear air and sees the ground, visual cues quickly override misleading vestibular inputs, and world coordinates are established by basic neural circuits of the brain. Conscious thought and special training are not required for this reorientation.
The need to fly in low or no visibility situations led to the development of a basic set of instruments (i.e., turn and bank indicator, altimeter, compass and airspeed indicator). Level flight is achievable by using the rudder to set the needle of the turn and bank (or needle-ball) indicator to center, using the ailerons to set the ball in the center, and then adjusting the elevators in response to both the altimeter (altitude increasing or decreasing) and airspeed (increased in a descent, decreased in a climb). Precise flight in one direction is obtainable by reference to the compass, which also provides additional rate of turn information. Generating spatial orientation by observing a set of instruments requires long training, much of it in actual flight.
These instruments fail to provide many of the direct basic visual cues which are the dominant source of orientation information for humans. This paucity of spatially orienting cues combined with vestibular and other signals that are often in conflict with each other and with reality are contributing causes for disorientation in instrument flight. In particular, fluid in the semicircular canals only responds to rotational acceleration, and the vestibular sacs only respond to linear acceleration. Therefore the middle ear cannot respond to constant velocity motion. Hence a constant rate of rotation around any axis is not sensed, and, without other inputs, the subjective sensation is that of flight in a straight line. If an aircraft is banked during a turn so that centrifugal force is balanced by gravitational force, the subjective sensation is that the plane is in not only straight flight but level flight. Thus, pilots are taught to disregard the xe2x80x9cseat of the pantsxe2x80x9d or vestibular and proprioceptive inputs, and to xe2x80x9cfly the instrument panelxe2x80x9d by synthesizing a mental image of attitude based on the instruments. Such synthesis is increasingly difficult to do in the presence of fatigue, injury, illness (including airsickness) or fear.
The gyro-horizon partially overcomes fundamental perception problems. The gyro-horizon provides a very small fixed (relative to the instrument panel) representation of the aircraft and an artificial horizon that moves behind the representation in response to a mechanical or electronic gyro, maintaining an orientation parallel to the actual horizon. The gyro-horizon presents the relative positions of the plane and the earth in pictorial form to provide a visual cue that can, with lengthy training, override vestibular inputs. The extensive training required demonstrates the gyro-horizon""s failure to completely solve the basic problems. Additionally, studies of instrument flight demonstrate that the best pilots scan and inter-compare all instruments related to instrument flight, while the less efficient or less experienced pilots tend to fix on the gyro horizon.
xe2x80x9cHeads up displaysxe2x80x9d (xe2x80x9cHUDsxe2x80x9d) have been developed to provide positional information which appears directly in front of the pilot. The images include optically collimated virtual images of instrument data arranged so that the eyes focus at optical infinity. In theory, this should provide a horizon that not only superimposes on the real horizon when it is visible, but is arranged so that the two superimpose when the latter is not visible. Many accidents are attributed to these displays. The problem appears to be a tendency of the pilot to fixate on a very small area rather than focusing at infinity, thus being visually distracted by objects around the display.
Instances in which pilots fail to react rapidly to ground approach or other audible warnings are well known. Typically, the pilot""s first reaction is to mentally question whether the warning is correct. Visual cues, such as seeing the ground rapidly approach, elicit an instant reaction.
Recent studies demonstrate that one of the strongest cross modal information exchanges is between the processing of vestibular and visual information, with visual information quickly overriding vestibular signals. For example, when a pilot breaks out of a cloud cover at night, remarkably few visual signals provide complete orientation. These may consist of a few lights on the ground, which need not be on the horizon. The reorientation does not require any set of eye movements or fixations, and the pilot may not consciously focus on anything. A world gestalt is subconsciously acquired. Unfortunately, vestibular inputs can have lasting effects, especially if they occur over a prolonged period before the canceling visual cues. Resulting dizziness and nausea can persist long after visually-induced reorientation occurs. Thus, it is important that disorientation be prevented or minimized. Experimental literature suggests that peripheral visual cues are extremely important in overcoming vestibular and proprioceptive perception. Studies also suggest that some of the strongest visual cues are those involving the movement of images across the retina. Much of this information is processed directly in the retina of the eye, and is transmitted to the brain as a set of signals separate and distinct from those transmitting images. Thus the brain is equipped to respond to the movement of objects in a field-of-view much better than to dial-based indications of rate.
One of the problems with the gyro-horizon is the size of the display, and its presentation. The angle subtended by the display is only approximately 10xc2x0, and no controlling positional visual cues approximating those in nature are presented. A second problem is that one must look consistently at the instrument rather than employing peripheral vision. A third problem is that a pilot is not automatically trained in instrument flight during routine flight, i.e., the instrument output is not continuously a normal part of his perception of world coordinates and is therefore not self teaching. A fourth problem is that the instrument panel itself does not change in attention attracting ways that are proportional to the extent of departure from normal straight and level flight. Given the large number of aircraft accidents attributed to pilot disorientation, and the large percentage of these which are related to instrument use, there is a need to develop an instrument display system which will give xe2x80x9cclear dayxe2x80x9d visual inputs under instrument flight conditions, i.e., inputs which are native to, and are automatically and correctly interpreted even by the untrained human cortex, and thus are understood intuitively, requiring a minimum of instruction.
This invention generally relates to providing visual cues that simulate real world visual cues in an attention attracting manner, such as through the use of movement and peripheral vision, to rapidly and effectively override vestibular signals relating to position, rate of turn, rate of descent and balance.
In one aspect, a flight information visualization system for use in an aircraft includes a first set of visual indicators extending generally vertically with respect to an interior of the aircraft, the first set of visual indicators including a first static reference indicator, at least a number of the visual indicators on either side of the first static reference indicator being selectively actuable to produce a visual indication; a second set of visual indicators extending generally vertically with respect to the interior of the aircraft and laterally spaced across a pilot""s field-of-view from the first set of visual indicators, the second set of visual indicators including a second static reference indicator, at least a number of the visual indicators on either side of the second static reference indicator being selectively actuable to produce a visual indication; and at least one processor coupled to selectively activate at least some of the visual indicators of the first and the second set of visual indicators, the processor activating one of the visual indicators in the first set of visual indicators spaced from the first static indicator in a direction and by a distance proportional to a distance between an actual horizon on a first side of the aircraft during a current set of flight conditions and a reference horizon on the first side of the aircraft for a nominal set of straight and level flight conditions as a most distal activated one of the visual indicators in the first set of visual indicators from the first static indicator and activating one of the visual indicators in the second set of visual indicators spaced from the second static indicator in a direction and by a distance proportional to a distance between the actual horizon on a second side of the aircraft during the current set of flight conditions and the reference horizon on the second side of the aircraft for the nominal set of straight and level flight conditions as a most distal activated one of the visual indicators in the second set of visual indicators from the second static indicator.
In another aspect, a peripheral visualization system for displaying flight information in an aircraft includes a first set of visual indicators extending along one approximately vertical side of a windshield of the aircraft including a first level flight visual indicator; a second set of visual indicators extending along another approximately vertical side of the windshield of the aircraft, including a second level flight visual indicator, the first and the second static visual indicators in approximate registration with a reference horizon in a field-of-view from the aircraft, the visual indicators of the first and the second sets of visual indicators selectively activated such than an imaginary line extending between an outermost activated one of the indicators from each of the first and the second sets of visual indicators is in approximate registration with an actual horizon in a defined field-of-view through the windshield of the aircraft; and a third set of visual indicators extending along one approximately horizontal side of the windshield of the aircraft, including a centerline visual indicator, the visual indicators of the third set of visual indicators selectively activated such that a distance between an outermost activated one of the third set of visual indicators on either side of a centerline is proportionate to a rate of turn of the aircraft.
In another aspect, a peripheral visualization system for displaying flight information in an aircraft includes means for providing a first static visual indication within a predefined field-of-view from the interior of the aircraft; means for providing a second static visual indication within the predefined field-of-view from the interior of the aircraft, the second static visual indication spaced horizontally across the field-of-view from the first static visual indication; means for providing a most distal first dynamic visual indication spaced vertically from the first static visual indication by a distance proportionate to a distance between an actual horizon on a first side of the field-of-view from the interior of the aircraft during a current set of flight conditions and a reference horizon on the first side of the field-of-view from the interior of the aircraft for a nominal set of straight and level flight conditions; and means for providing a most distal second dynamic visual indication spaced vertically from the second static visual indication by a distance proportionate to a distance between an actual horizon on a second side of the field-of-view from the interior of the aircraft during a current set of flight conditions and a reference horizon on the second side of the field-of-view from the interior of the aircraft for a nominal set of straight and level flight conditions.
In still another aspect, a method of providing visual flight information in an aircraft having an interior includes providing a first static visual indication within a predefined field-of-view from the interior of the aircraft; providing a second static visual indication within the predefined field-of-view from the interior of the aircraft, the second static visual indication spaced horizontally across the field-of-view from the first static visual indication; providing a most distal first dynamic visual indication spaced vertically from the first static visual indication by a distance proportionate to a distance between an actual horizon on a first side of the field-of-view from the interior of the aircraft during a current set of flight conditions and a reference horizon on the first side of the field-of-view from the interior of the aircraft for a nominal set of straight and level flight conditions; and providing a most distal second dynamic visual indication spaced vertically from the second static visual indication by a distance proportionate to a distance between an actual horizon on a second side of the field-of-view from the interior of the aircraft during a current set of flight conditions and a reference horizon on the second side of the field-of-view from the interior of the aircraft for a nominal set of straight and level flight conditions.
In a further aspect, a computer readable media stores instructions for causing a computer to display flight information on a number of indicators in an aircraft, by: providing a first static visual indication within a predefined field-of-view from the interior of the aircraft; providing a second static visual indication within the predefined field-of-view from the interior of the aircraft, the second static visual indication spaced horizontally across the field-of-view from the first static visual indication; providing a most distal first dynamic visual indication spaced vertically from the first static visual indication by a distance proportionate to a distance between an actual horizon on a first side of the field-of-view from the interior of the aircraft during a current set of flight conditions and a reference horizon on the first side of the field-of-view from the interior of the aircraft for a nominal set of straight and level flight conditions; and providing a most distal second dynamic visual indication spaced vertically from the second static visual indication by a distance proportionate to a distance between an actual horizon on a second side of the field-of-view from the interior of the aircraft during a current set of flight conditions and a reference horizon on the second side of the field-of-view from the interior of the aircraft for a nominal set of straight and level flight conditions.
In yet a further aspect, a method of providing visual flight information in an aircraft having an interior includes providing a first set of visual indications extending generally vertically with respect to an interior of the aircraft, the first set of visual indications including a first static reference indication and a most distal indication spaced from the first static indication in a direction and by a distance proportional to a distance between an actual horizon on a first side of the aircraft during a current set of flight conditions and a reference horizon on the first side of the aircraft for a nominal set of straight and level flight conditions; a second set of visual indications extending generally vertically with respect to the interior of the aircraft and laterally spaced across a pilot""s field-of-view from the first set of visual indications, the second set of visual indications including a second static reference indication and a most distal indication spaced from the second static indication in a direction and by a distance proportional to a distance between the actual horizon on a second side of the aircraft during the current set of flight conditions and the reference horizon on the second side of the aircraft for the set of straight and level flight conditions; and providing a third set of visual indications extending generally horizontally with respect to the interior of the aircraft, the third set of visual indications including a third static reference indication and a most distal third visual indication on either side of the third static indication, spaced from the third static indication by a distance proportional to a rate of turn of the aircraft.
In still a further aspect, a computer readable media stores instructions for causing a computer to display flight information on a number of indicators in an aircraft, by: providing a first set of visual indications extending generally vertically with respect to an interior of the aircraft, the first set of visual indications including a first static reference indication and a most distal indication spaced from the first static indication in a direction and by a distance proportional to a distance between an actual horizon on a first side of the aircraft during a current set of flight conditions and a reference horizon on the first side of the aircraft for a nominal set of straight and level flight conditions; a second set of visual indications extending generally vertically with respect to the interior of the aircraft and laterally spaced across a pilot""s field-of-view from the first set of visual indications, the second set of visual indications including a second static reference indication and a most distal indication spaced from the second static indication in a direction and by a distance proportional to a distance between the actual horizon on a second side of the aircraft during the current set of flight conditions and the reference horizon on the second side of the aircraft for the set of straight and level flight conditions; and providing a third set of visual indications extending generally horizontally with respect to the interior of the aircraft, the third set of visual indications including a third static reference indication and a most distal third visual indication on either side of the third static indication, spaced from the third static indication by a distance proportional to a rate of turn of the aircraft.
The display system described may be used in other applications such as in ships at sea, naval landing craft, and in enclosed military vehicles driving over rough terrain to both help prevent seasickness and to assist in navigation.
The systems described may be incorporated into a computer program to display a programmable external environment, the interior of an aircraft of vessel as seen by a pilot or observer, flight or other instruments, and the display of the present invention, which program is controllable by an operator or subject in such a manner as to mimic flight or other conditions. The program may be written in such a manner as to present a series of conditions requiring measurable operator input. The system and program may be further designed to measure and record the time intervals required, to make those responses, and to objectively evaluate the appropriateness of the responses. This flight simulator will allow experimental studies to be done in which the subject passes, in simulation, through VFR and IFR conditions, and detects and responds to programmed attitudinal changes with only the standard instrument display visible, with only the display of the present invention visible, with both visible, and with neither visible. Using subjects with no flight training, subjects with only VFR training, and subjects with extensive instrument flight training, the ease with which situational awareness may be acquired, and the speed and appropriateness of responses may be measured.
In an additional aspect, a flight information display system for providing flight information in an aircraft includes a display; a static horizon reference indicator; and a processor coupled to the display and configured to produce an image on the display including a number of substantially parallel pitch lines, at least a first one of the pitch lines having a first visual characteristic and at least a second one of the pitch lines displayed at a same time as the first pitch line having a second visual characteristic, different from the first visual characteristic, a distance between the horizon reference indicator and an interface between the first and the second pitch lines being proportional to a pitch of the aircraft under current flight conditions, where the pitch lines scroll perpendicularly across the display at a rate proportional to a rate of altitude change of the aircraft, if any.
In still another additional aspect, a method of providing a display of flight information for an aircraft on a display includes receiving a set of flight information for the aircraft; determining a number of successive images based on the received flight information, the images including a number of substantially parallel pitch lines, at least a first one of the pitch lines having a first visual characteristic and at least a second one of the pitch lines having a second visual characteristic in the same image, different from the first visual characteristic, a distance between a horizon reference indicator and an interface between the first and the second pitch lines being proportional to a pitch of the aircraft under current flight conditions; and displaying the determined images.
In still a further additional aspect, a computer readable media stores instructions for causing a computer to display flight information on a number of indicators in an aircraft, by: displaying a number of substantially parallel pitch lines, at least a first one of the pitch lines having a first visual characteristic and at least a second one of the pitch lines displayed at a same time as the first pitch line having a second visual characteristic, different from the first visual characteristic, a distance between a horizon reference indicator and an interface between the first and the second pitch lines being proportional to a pitch of the aircraft under current flight conditions, where the pitch lines scroll perpendicularly across the display at a rate proportional to a rate of altitude change of the aircraft, if any; and displaying a number of substantially parallel yaw lines, the yaw lines substantially perpendicular to the pitch lines, where the yaw lines scroll perpendicularly across the display with respect to a static heading reference indicator at a rate proportional to a rate of turn of the aircraft, if any.